A computer program may be executed by sending an ordered sequence of instructions to a central processing unit (CPU). A computer may contain a number of CPUs. Each CPU may contain a number of execution cores. On a computer with a single CPU with a single core, an operating system may allow the creation of a number of execution threads to simulate the ability to execute a number of programs at the same time. There are a number of ways of mediating access to the CPU for the different execution threads. One way is for the operating system to allow each execution thread to execute a number of instructions before it is preempted by another thread which is then allowed to execute the same number of instructions. On systems with more than one CPU or more than one execution core per CPU, it is possible to use the same mechanism to allow a greater number of threads than available cores.
When there is more than one execution core, two different threads may be executed at exactly the same time. If two or more threads are accessing the same piece of data, simultaneous execution may lead to unexpected results. Similar problems may also occur when there is only one execution core if threads are preempted in such a way that one accesses data to which another thread expects to have exclusive access.
A computer program may split tasks of different character into different execution threads. In so doing it is important to ensure that the different threads synchronize their access to memory. A computer process may be split into two threads in such a way that one thread outputs packets of data to be processed by the second thread. Without a buffer in between, both threads would be limited to execute at the rate of the slowest thread. When implementing a buffer between the threads it is necessary to ensure that the threads' access to the buffer is synchronized in such a way that there is no data loss.
A buffer can be implemented as a linear queue of objects, where the first thread inserts an object into one end of the queue and the second thread pulls objects from the opposite end of the queue.
When two threads access the queue simultaneously a number of problems can occur. If the first thread overwrites an object in the queue before the second thread has processed it, data loss may occur. Secondly, if objects used to carry data are not recycled properly, memory leaks and memory fragmentation may cause a serious reduction in performance.
One method of preventing inconsistent access to the queue object is to use programmatic locks. A lock enables a thread to obtain exclusive access to a section of memory for a period of time. For example, the first thread could lock the queue, insert an object, and unlock it. During this period, the second thread would not be able to access the queue. Similarly, the second thread would lock the queue, remove an object from the queue, and unlock it. This approach removes some of the benefits of multi-threading, and can cause significant performance penalties.
One example of a computer program where one thread passes information to a second thread is a web server. A web server may contain at least one worker thread and at least one logging thread, where the working thread receives and responds to Hypertext Transfer Protocol (HTTP) requests and the logging thread produces a log with summary information about each request and its respective response. The worker thread may package information about a request and its response into an object and pass it to the logging thread. The logging thread may then write the corresponding log entry to disk.
It will take the worker thread a number of CPU cycles to produce a response to the HTTP request; the information about the request relevant for the log may accordingly be available at an earlier stage than the information about the response. In order to aggregate the information about the request and the response into a single log entry the two must at some stage be aggregated. This may have an impact on performance.